IN THE CLAIMS 



1 . (Currently amended) A method for providing collateral information for 
inclusion with an information stream, comprising steps of: 



examining the information 
occur in the information 



automatically generatin 
and 



stream to recognize a presence of events that 
stream; 



database queries from said recognized events; 



analyzing results of said database qu e ry r e su l ts queries so as to rank and 
select d a tabas e qu e ry said results to be inserted into the information 
stream as co ll ater al information that is collateral to said recognized 
events . 

2. (Original) A methdd as in claim 1 , wherein the step of analyzing 
comprises a step of ranking the database query results based on a plurality of 
criteria. 



3. (Original) A method as in claim 2, wherein the plurality of criteria 
comprise a score derived from a free text search of the database using text that 
is automatically extracted from the information stream, on a number of named 
entities appearing in the t ?xt and in the database query results, and on a 
taxonomy path score, where the taxonomy path score represents an amount of 
relatedness between a ta <onomy-related information element found in the text 
and a predetermined taxc nomy tree. 

4. (Original) A method as in claim 1, wherein the database queries are 
automatically generated based on information corresponding to a list that 



identifies topics in text that is automatically extracted from the information 
stream, where the topics correspond to predetermined topic taxonomies. 

5. (Original) A method as in claim 1, wherein the step of examining 
comprises a step of automatically Extracting text from the information stream. 

6. (Original) A method as inj claim 5, wherein the step of examining further 
comprises steps of: 

segmenting the text into sentences; and 

operating on the sentences to identify topics that correspond to 
predetermined topic taxonomies, wherein the step of automatically 
generating database queries operates on identified topics. 

7. (Original) A method as in claim 5, wherein the step of automatically 
extracting text from the infornr/ation stream comprises a step of operating a voice 
recognition system. 

8. (Original) A method as in claim 5, wherein the step of automatically 
extracting text from the information stream comprises a step of extracting closed 
caption text. 



9. (Original) A method as in claim 5, wherein the step of automatically 
extracting text from the information stream comprises a step of operating a 
character recognition system. 

10. (Original) A method as in claim 5, wherein the step of automatically 
extracting text from the information stream comprises a step of also generating 
text that is descriptive of a number of human faces that are present in an image 
conveyed by the information stream. 



3 



1 1 . (Original) A method as in claimf 5, wherein the step of examining the 
information stream further comprises steps of segmenting the text into sentences 
and a step of operating on the sentences to identify topics that correspond to 
predetermined topic taxonomies and the presence of names of entities, and 
further comprising steps of assembling/a list comprised of an identified topic 
having a start time and an end time, as well as any named entities that occur 
between the start time and the end time, assembling a query object comprised of 
named entities that occur between the start time and the end time of the 
identified topic, searching at least on* database to identify a first set of stored 
documents that correspond to the topic, identifying a subset of the first set of 
documents that contain the named entities, identifying a second set of 
documents that correspond to woras found in the text; scoring the returned 
documents based on a plurality of/criteria and ranking the documents based on 
their scores. / 

12. (Original) A method for providing collateral information for multiplexing 
with an information stream, comprising steps of: 

converting the information stream into text; 

analyzing the text to identify information elements; 

automatically generating queries from the information elements for 
searching at least one database; 

extracting data from /database search results that is relevant to the 
information stream; and 

multiplexing the data into the information stream for presentation at a 
destination of the information stream. 
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13. (Original) A method as in claim 12/wherein the step of extracting 
comprises a step of ranking the extracted information based on a plurality of 
criteria, and where the step of multiplexing juses the ranked data. 

14. (Original) A method as in claim/12, wherein the step of extracting 
comprises a step of ranking extracted document information based on a score 
derived from a free text search of a document database using the text, on a 
number of named entities extracted from the text that are found in the 
documents, and on a taxonomy path spre, where the taxonomy path score 
represents an amount of relatedness Between a taxonomy-related information 
element identified in the text and a predetermined taxonomy tree. 

15. (Original) A method as in/claim 12, wherein the queries are generated 
based on information elements thau correspond to a list of information elements 
identifying topics in the text being analyzed, where the topics correspond to 
predetermined topic taxonomies. / 

16. (Original) A method as in claim 12, wherein the step of analyzing the 
text comprises steps of segmenting the text into sentences and a step of 
operating on the sentences to identify topics that correspond to predetermined 
topic taxonomies, and wherein/the step of automatically generating queries 
operates on identified topics. / 

17. (Original) A method as in claim 12, wherein the step of analyzing the 
text comprises steps of at least segmenting the text into sentences, identifying 
names of entities within the/text, and a step of operating on the sentences to 
identify topics that correspond to predetermined topic taxonomies, and wherein 
the step of automatically Generating queries operates on identified topics and 
ranks the database searcfh results based at least on numbers of named entities 
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found and on an amount of relatedness between a taxonomy-related information 
element identified in the text and a predetermined taxonomy tree. 

18. (Currently amended) A system fpr providing collateral information for 
inclusion with an information stream, saidjsystem operating in real time or 
substantially real time and comprising: 

a subsystem for examining the information stream to recognize a 
presence of events that occur in khe information stream; 

a subsystem, having an input aoupled to an output of said examination 
subsystem, for automatically generating database queries from said 
recognized events; 

a database for receiving sa/d database queries; and 

a subsystem, having an input coupled to an output of said database, for 
analyzing results of said database qu e ry r e su l ts gueries so as to rank and 
select databas e qu e ry sa/id results to be inserted into the information 
stream as co lla t e r al information that is collateral to said recognized 
events . 

19. (Original) A systefn as in claim 18, wherein the analyzing subsystem 
employs ranking criteria comprised of a score derived from a free text search of 
the database using text that is automatically extracted from the information 
stream, on a number of named entities appearing in the text and in the database 
query results, and on a taxonomy path score, where the taxonomy path score 
represents an amount of relatedness between a taxonomy-related information 
element found in the text and a predetermined taxonomy tree, and wherein the 
query generation subsystem generates queries based on information 
corresponding to a Iis4 that identifies topics in the text that is automatically 
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extracted from the information stream, where the topics correspond to elements 
of the taxonomy tree. 

20. (Original) A system as. in claim 18, wherein said examining subsystem 
comprises at least one unit for automatically extracting text from the information 
stream, a unit for segmenting thye text into sentences and at least one unit for 
operating on the sentences to identify topics that correspond to predetermined 
topic taxonomies, wherein said query generation subsystem automatically 
generates database queries based at least in part on identified topics. 



21 . (Original) A system^ as in claim 20, wherein said text extracting unit 
comprises at least one of a voice recognition system, a system for extracting 
closed caption text, and a character recognition system. 



22. (Original) A systerp 
comprises a unit for 
faces that are present in an 



as in claim 20, wherein said examining subsystem 
generating text that is descriptive of a number of human 
mage conveyed by the information stream. 



23. (Original) A syste m as in claim 20, and further comprising a unit for 



operating on the sentences 
further comprising a unit for 
having a start time and an 
between the start time and 



to identify the presence of names of entities, and 
assembling a list comprised of an identified topic 
end time, as well as any named entities that occur 
he end time, and where the query generation 
subsystem assembles a qiiery object comprised of named entities that occur 
between the start time and the end time of the identified topic for searching said 
database to identify a first set of stored documents that correspond to the topic, a 
subset of the first set of dc cuments that contain the named entities, a second set 
of documents that corresp ond to words found in the text; and where said 
analyzing subsystem scores the returned documents based on a plurality of 
criteria and ranks the documents based on their scores. 
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24. (Currently amended) A compyter-implemented method for generating 
collateral information from an audio/video stream, comprising steps of: 

examining the audio/video stream to recognize a presence of events that 
occur in the audio/video strear 

generating database queries from said recognized events; and 

analyzing results of said database qu e ry r e su l ts queries so as to rank and 
select databas e query said results to be presented as th e co ll at e r al 
information that is collateral to said recognized events . 

25. (Previously presented) A method as in claim 24, and further 
comprising a step of inserting trae collateral information into the audio/video 
stream in real time or substantially real time. 

26. (Previously presented) A method as in claim 24, wherein the step of 
examining includes a step of generating a speech transcript from at least the 
audio portion of the audio/visjual stream, and wherein recognized events 
comprise speech topics. 

27. (Previously presented) A method as in claim 24, wherein the 
audio/video stream originates as a television broadcast signal. 

28. (Previously presented) A method as in claim 24, wherein the 
audio/video stream originates at a meeting, and further comprising a step of 
presenting the collateral ipformation to meeting participants in real time or 
substantially real time. 



29. (Previously presented) A method as in claim 28, wherein the step of 
presenting comprises a/step of inserting the collateral information into the 
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audio/video stream, and displaying the pudio/video stream to the meeting 
participants. 

30. (Previously presented) A rihethod as in claim 28, and further 
comprising a step of archiving at least the collateral information. 

31. (Previously presented) A method as in claim 24, wherein the database 
queries are automatically generated based on information corresponding to 
identified topics extracted from thfe audio/video stream, where the topics 
correspond to predetermined topic taxonomies. 

32. (Previously presented) A method as in claim 24, wherein the step of 
examining includes steps of generating a speech transcript comprised of words 
from at least the audio portion/of the audio/video stream; segmenting the words 
into sentences; and operating on the sentences to identify topics that correspond 
to predetermined topic taxonpmies, wherein the step of generating database 
queries operates on identified topics. 



33. (Original) A computer readable media having recorded thereon a 
program for providing collateral information for inclusion with an information 
stream, the program comprising instructions for examining the information stream 
to recognize a presence pf events that occur in the information stream, for 
automatically generating database queries from recognized events; and for 
analyzing database qu6ry results so as to rank and select database query results 
to be inserted into the information stream as collateral information. 
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